Avian detection systems and methods

ABSTRACT

Provided herein are detection systems and related methods for detecting moving objects in an airspace surrounding the detection system. In an aspect, the moving object is a flying animal and the detection system comprises a first imager and a second imager that determines position of the moving object and for moving objects within a user selected distance from the system the system determines whether the moving object is a flying animal, such as a bird or bat. The systems and methods are compatible with wind turbines to identify avian(s) of interest in airspace around wind turbines and, if necessary, take action to minimize avian strike by a wind turbine blade.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/832,370, filed Aug. 21, 2015, which claims the benefit of U.S. PatentApplication No. 62/040,018, filed Aug. 21, 2014, each of which areincorporated by reference in its entirety.

BACKGROUND OF INVENTION

There is an interest and need in the art for reliable and robustdetection of flying avians. Avian detection systems have manyapplications, ranging from avian counts, classification and/oridentification in a specific geographical location, to deterrence andcounter-measure systems for aviation and wind production systems. Acommon objective of such systems is the replacement of subjective andinaccurate human-based counts with an automated and reliable detectionsystem. This is a reflection that human-based detection of flying aviansrequires intensive training to be able to properly identify avians andspecies thereof, is highly labor intensive and is inherently inaccurate.

One specific application of bird detection systems is for wind energygeneration. There is concern as to the risk to avians arising fromavian-wind turbine collision. One challenge for accurately assessing therisk of wind turbine collision by a flying avian is the difficulty inreliably determining the number of birds and the species of such birdsin an area of a turbine or a to-be-located turbine. It is difficult tocontinuously monitor airspace, and so conventional bird strike fatalitysearches are conducted using systematic schedules with an attendantestimate of fatalities based on a uniform distribution over time, asexplained in “Impacts of Wind Energy Facilities on Wildlife and WildlifeHabitat” Technical Review 07-2. September 2007 (available at:wildlife.org/documents/technical-reviews/docs/Wind07-2.pdf). This hasnumerous disadvantages, including not accounting for cluster fatalities,injured avians that leave the immediate area or are removed byscavengers, and the challenge associated with reliably and consistentlylocating carcasses. Regardless of such inaccuracies, there has beendocumentation of raptor fatalities at wind turbine fatalities. See,e.g., Id. at p. 15 and references cited therein, including forCalifornia-based wind-farm facilities such as the Altamont Pass WindResource Areas (APWRA), San Goronio and Tehachapi. Estimates for raptorkills at APWRA per year range from between 881-1300 or about 1.5-2.2raptor fatalities/MW/year, including about 75 to 116 Golden Eagles. Withthese statistics in mind, there is interest in bird detection systemsincluding for use with wind-farm planning, development, expansion andoperation.

One example of a bird detection and dissuasion system is Dtbird® byLiquen (description available atdtbird.com/index.php/en/technology/detection). A fundamental limitationof that system is the reported detection efficiency of 86-96% for adistance of only 150 m from the wind turbine, with an efficiency thatfalls off with increasing distances.

Other implementations of avian detection systems are based on radarincluding, for example, Merlin Avian Radar Systems by DeTect(www.detect-inc.com/avian.html). Those systems, however, require bulkyand expensive radar equipment and are not suited to distinguishingbetween avian species of interest. For example, a fundamental drawbackis the inability to distinguish between an endangered or valued raptorspecies and another bird species that is neither endangered or ofcommercial importance. For example, it would be beneficial todistinguish between a golden eagle and a turkey vulture, for examplewith action implementation for wind blade speed tailored to species ofinterest only. Radar systems are not suited for such applications, asthey do not obtain visual details that would otherwise distinguishbetween different bird species that are similarly sized and/or havesimilar flight characteristics. Furthermore, radar-based systems producemany false-positives, including arising from moving objects such as aturbine blade.

U.S. Pat. Pub. 2013/0050400 (Stiesdal) describes an arrangement toprevent collision of a flying animal with a wind turbine. Stiesdal,however, is limited in that there is not full spatial coverage, butinstead focuses on imaging horizontal directions. U.S. Pat. No.8,598,998 describes an animal collision avoidance system. Other systemsare described, for example, in U.S. Pat. Pub. Nos. 2009/0185900(Hirakata) and 2008/0298692 (Silwa). Each of those systems have inherentlimitations, such as not providing full coverage of all directions ofthe surrounding airspace, do not provide sufficient detection efficiencyand/or cannot reliably distinguish between avian species and confinedetection to a specific avian species.

Because of the risk to migratory birds, raptors and other avians ofinterest including bats, it is desirable to have a reliable,cost-effective and robust system for identifying certain avian species,including before siting of wind turbine(s) as well as during windturbine operation. Provided herein are various methods and systems foravian detection, including highly reliable and sensitive detectionsystems over sufficiently large detection ranges that provide sufficienttime to take action to minimize or avoid unwanted contact between aspecific avian species and the wind turbine, while minimizingunnecessary wind turbine shutdown for avian species or other movingobjects that are not of interest, while avoiding the need for largegroups of human observers.

SUMMARY OF THE INVENTION

The disclosed systems and methods provide detection of a flying avianfor large airspace volumes in a manner that is both cost-effective andreliable. The systems are completely scalable, being compatible with anynumber of imagers and systems, dependent on the application of interest,with larger areas covered by increasing the number of systems.Integration of specially configured imagers with efficient algorithmsfacilitate rapid and accurate determination of moving objects along withwhether such moving objects may represent an avian of interestwarranting further analysis for moving objects within a user-definedairspace. A first wide field of view imager assists with simultaneouslymonitoring a very large airspace and images any number of potentialmoving objects. Various algorithms, including pattern recognition, edgedetection and boundary parameter analysis, and behavior analysis ofavian body position and posture or perspective relative to theenvironment, further refines the decision as to whether a detectedmoving object should be further analyzed. A second high zoom imager,such as a stereo imager, optically zooms on relevant detected movingobjects and can provide rapid information as to the distance of themoving object and additional information related to finer opticalcharacteristics of the moving object to facilitate speciesidentification of a flying avian.

One advantage of the systems provided herein is the ability to imagesurrounding airspace in all available viewing directions from a sourceor origin centered on or around the systems. This ability to imageall-views from a system to the surrounding airspace is generallyreferred herein as providing substantially complete hemisphericalcoverage of the surrounding airspace. The configuration of the imagersand integration with a processor that analyzes images facilitatesreliable detection at a large distance for any viewing direction, suchas greater than 600 m and up to at least about 1.2 km, and any rangestherein. The ability to reliably detect a flying avian at such largedistances is particularly useful for wind turbine systems where a fastdiving or flying raptor requires a sufficiently advanced detection andwarning to permit action implementation ahead of impact. For example,reliable detection at a range of between about 800 m to 1 km isbeneficial for providing sufficient stop time for a moving wind turbineblade before a speeding avian would otherwise potentially contact amoving wind turbine blade. Furthermore, the large airspace coveragereduces the total number of systems required, with one system providingreliable airspace coverage that may otherwise require a plurality ofconventional systems. This is a reflection of the capacity of theinstant systems for collection, storage, and/or analysis of largevolumes of data, including simultaneously.

The avian detection system may be for detecting a flying avian in anairspace. The system comprises a first imager having a wide field ofview for detecting a moving object; a second imager having a high zoom;a positioner operably connected to the second imager for positioning thesecond imager to image the moving object detected by the first imager;and a processor operably connected to receive image data from the firstimager, the second imager, or both to identify a moving object that is aflying avian based on image data. An advantage of the instant detectionsystems is the capability of substantially complete hemisphericalcoverage of airspace surrounding the avian detection system up to largedistances from the system.

Any of the systems described herein may comprise a plurality of firstimagers and second imagers arranged in a spatial configuration toprovide substantially complete hemispherical coverage.

The first imager may comprise a fish-eye lens or detector configured toimage visual data from a substantially hemispherical surroundingairspace, and may include a plurality of individual images to providethe desired field-of-view.

The substantially complete hemispherical coverage may provide coveragefor a volume of airspace having a detection distance from the firstimager that is greater than or equal to 0.6 km and less than or equal to2 km or between 0.6 km and 1.2 km. With this in mind, any of theairspaces provided herein may have a volume associated therewith fromwhich a corresponding half-hemisphere radius is determined (e.g.,V_(airspace)=(⅔)πr³, where r is selected so as to provide the airspacevolume equivalent to that being monitored by the system). Accordingly, rprovides a type of average detection distance that is effectively imagedby any of the systems provided herein. Variation in r over the airspacevolume outer surface may be statistically quantified, such as by astandard deviation, standard error of the mean, or the like. In anaspect, the standard deviation is less than or equal to about 20%, 10%or 5% of an average value of r. For stand-alone systems that do notdirectly observe airspace immediately above the system, a second systempositioned at a separation distance may provide the desired coverage ofthat airspace, so that in combination substantially or completehemispherical coverage around the system is achieved.

The systems and methods provided herein may be described in terms ofdetection efficiency for a selected avian species of interest that isgreater than 96% for the volume of airspace, including better than 99%or 99.9% so that there is a statistically insignificant chance ofmissing an avian species of interest. The systems and methods providedherein may be described as having a percentage of false positives for aflying avian species of interest that is less than or equal to 5% forthe volume of airspace. The detection efficiency, along with low levelof false positive identification, is a fundamental improvement over theart, particularly considering the large volumes of airspace that aremonitored, such as between about 0.45 km³ and 16.8 km³ or 0.45 km³ and2.1 km³ (corresponding to detection distances between about 0.6 km and 2km, or 0.6 km and 1 km, respectively), or any subrange thereof.

The avian species of interest may be a golden eagle or an endangeredflying avian species.

The processor may identify an output of a subset of pixels of the firstimager or the second imager corresponding to the moving object. Thesubset of pixels may comprise neighboring pixels, directly adjacentpixels, or both. The output of the subset of pixels may be an array ofintensity values, with each value corresponding to an individual pixelintensity and/or a color value, with various colors assigned a numericalvalue to assist with color identification. The output of the subset ofpixels may be a time varying output. In this manner, regions areidentified corresponding to a moving object.

The processor may analyze the output of the subset of pixels todetermine if the moving object is a flying avian. The output may furtherbe a single frame or may be from more than one frame, a time course of asingle frame or from more than one frame, or a combination thereof, tofacilitate a time-varying output.

The processor may analyze the output to identify the presence of one ormore threshold identification attributes, such as a thresholdidentification parameter that is a boundary parameter. The boundaryparameter may correspond to an edge boundary signature characteristic ofa flying avian. In this manner, the threshold identification parametermay provide an initial cut-off for determining whether to furtheranalyze or characterize the subset of pixels.

In an aspect, the edge boundary signature may be identified bydetermining an intensity gradient of the output of the subset of pixels.The edge boundary signature may be identified by comparing the intensitygradient to one or more reference values.

In this aspect, “reference values” may be used to distinguish objectsthat correspond to non-animal objects, such as clouds, debris, plants,or artificial objects. For example, the edge boundary signature maycorrespond to an edge straightness parameter, and the output identifiedas corresponding to an artificial object for an edge straightnessparameter indicative of an artificially constructed straight line.Straight lines or unduly smooth curves tend to be artificial in natureand may be used to assist with preliminary characterization of a movingobject as not a flying avian. Accordingly, the edge boundary signaturemay relate to quantification of a parameter related thereto, such as alength, curvature, smoothness, roughness, color, light gradient, lightintensity, light wavelength, uniformity, or the like.

In an aspect, the edge boundary signature corresponds to a flying avian,such as a threatened or endangered avian species of interest.

Any of the one or more threshold identification attributes may be a timeevolution parameter, such as a time evolution parameter corresponding toa time evolution signature characteristic of movement of a flying avian.

In an aspect, the one or more threshold identification attributes may bea color parameter. In an aspect, the color parameter may correspond to acolor signature characteristic of a flying avian.

Upon identification of the presence of one or more thresholdidentification attributes, the processor may analyze the output of thesubset of pixels to determine one or more avian identificationparameters.

The processor may compare the output of the subset of pixels to one ormore reference values in a reference image database to determine if themoving object is a flying avian, including assigning a probability thatthe moving object is a flying avian and/or a flying avian species ofinterest. In this manner, resources may be appropriately prioritized tothe higher probability objects.

The processor may compare output of the subset of pixels to referencevalues to determine one or more avian identification parameters selectedfrom the group consisting of size, speed, wing span, wing shape, avianposture or ratio of wing span width to height or vice versa (w/h orh/w), color, boundary shape, geometry, light intensity, and flighttrajectory. In this context, “reference values” may refer to values thatare empirically obtained from known flying avians. For example, a flyingavian may be observed and the size, speed, wing span, wing shape, color,boundary shape, geometry, intensity, posture and typical trajectoriesobtained and defined by ranges about an average. These parameters may beobtained for a specific avian or a plurality of avians. The referencevalues may be provided in a reference image database or determined usingone or more reference image algorithms, with the database or algorithmoperably connected to the processor. The reference image algorithm maybe part of a machine learning application so that the system ischaracterized as a smart system that continuously learns and updates tofurther improve avian characterization as more reference images areobtained and characterized.

In an aspect, the processor analyzes output of the subset of pixels viaa pattern recognition algorithm. The pattern recognition algorithm mayidentify the subset of pixels as a species of flying avian, including athreatened or endangered raptor species.

Any of the systems and methods provided herein may have a processor thatanalyzes output of the subset of pixels from a plurality of framescontaining the image data, wherein the subset of pixels spatially moveswith time (for a fixed-stationary imager) and the movement with time isused to determine a trajectory of the output of the subset of pixels. Inthis manner, the trajectory may comprise positions, distances,velocities, directions or any combination thereof over time.Accordingly, the systems and methods may further comprise determining apredictive trajectory corresponding to a future time interval. For thosesituations where an object is flying directly toward an imager, themovement may effectively be determined by an increase in number ofpixels in the output of the subset of pixels with time, as the objectmoves toward the imager. Similarly, for an object moving directly away,the number of pixels in the output of the subset of pixels with time maydecrease. A moving object that is not substantially changing in distancefrom the imager, may correspond to a subset of pixels that does notsignificantly change in number with time, but will, in contrast todirect flight to and away from an imager, have a change in pixellocation relative to a non-moving camera.

Any of the pattern recognition algorithms may comprise a database ofphysical parameters associated with a flying avian species of interest,and the processor compares a physical parameter determined from thefirst imager or the second imager to a corresponding physical parameterfrom the database of physical parameters to filter out moving objectsthat are not a flying avian or are not a flying avian species ofinterest and/or assign probabilities thereto. Such parameters are alsoreferred herein as an “avian identification parameter”. The avianidentification parameter is any observable parameter useful forclassifying a moving object as an avian, including a specific avianspecies. Examples include physical parameters of the avian, such assize, color, shape, or other physically distinctive characteristics.Other parameters include flight trajectory or wing motion (or lackthereof).

Any of the avian detection systems and methods may be used to detect aflying avian of interest that is a government, agency, federally orstate-protected raptor, such as an endangered raptor species or a goldeneagle.

Any of the avian detection systems utilize a processor that filtersmoving objects that do not correspond to an avian species of interest.For example, the avian may correspond to a plentiful species that is notendangered such as a turkey vulture, for example. Alternatively, themoving object may in fact not even be an avian, but instead debrisblowing through the airspace, an aircraft, cloud movement, or othernatural motion of vegetation. The systems provided herein accommodatesuch moving objects and, for such objects, no action implementation istaken. This is in contrast to radar-based systems that cannoteffectively ascertain such false positives.

In an aspect, the systems and methods are described further in terms ofan optical parameter of the imagers. For example, the first imager widefield of view may be quantified and selected from a range that isgreater than or equal to 0.5 km² and less than or equal to 1.6 km² at adefined detection distance, such as about 0.8 km to about 1.2 km.Alternatively, the first imager may be described as having a certainrange of the field of view. For a first imager having a rectangularlens, the fields of view may be described in a horizontal and a verticaldirection, such as independently selected between about 60° and 180°, orbetween about 60° and 120°. A first imager system (e.g. a wide field ofview or WFOV system) may be formed from a plurality of first imagers,such as a pair of imagers aligned relative to each other at a 60° to 70°angle that, in combination, provide an at least 120° reliable coverage.A combination of those first imager systems then can provide completecircumferential coverage and, up to a point, hemispherical coverage. Inan aspect, any of the imagers provided herein may be described as havinga resolution. As used herein, resolution refers to the ability toreliably resolve elements of a defined size. For example, the firstimager may have a resolution that is suitable to detect a moving objectthat is a bird. In an aspect, the resolution of the first imager capableof detecting a moving object that may be a bird is between about8″/pixel to about 14″/pixel. Similarly, the resolution of the firstimager may be about 0.3 m. Alternatively, the resolution of the firstimager may be described in functional terms as being of sufficientresolution to detect a bird of interest having a defined size, such asthe size of an avian of interest, including a golden eagle.

The second imager may be described, for example, as having a high zoomthat may be selected from a range that is greater than or equal to 10×and less than or equal to 1000×, or that may be fixed but at a highzoom, and may be also be described as part of a stereo imager to providedistance information. Similar to the resolution described for the firstimager, the second imager may be described in terms of a resolution. Inparticular, the second imager is configured to be able to provide a highzoom on a region identified, at least in part, by the first imager as amoving bird. The resolution is selected so as to provide information inconfirming the moving object is a bird and also for speciesidentification. In an aspect, the resolution of the second imager isgreater than or equal to 0.25 cm/pixel and less than or equal to 10cm/pixel, including greater than or equal to 0.25 cm/pixel and less thanor equal to 1 cm per pixel. At this high resolution, precise identifyingfeature information may be obtained for the moving object, down to eyecolor, beak color, ruffling shape, tail feather shape, wing tip shape,and other visually distinctive shapes for the avian species of interest.The “high zoom” may simply refer to the higher resolution compared tothe first imager, with a fixed high zoom used in combination with apositioner such as a pan and tilt, to ensure the second imager images adesired region identified by the first imager.

To provide field of view to detect an avian positioned anywhere withinthe airspace surrounding the imaging system, a plurality of firstimagers may be arranged in distinct alignment directions to provide full360° and hemispherical coverage by the plurality of first imagers fieldsof view up to and including a vertical alignment direction. In thisaspect, one of the first imagers is arranged in a vertical alignmentdirection to provide coverage for airspace in a vertical direction thatis not otherwise covered by another first imager field of view. This isparticularly relevant for airspace that is around a physical objectextending a vertical height, such as a building, a vehicle, or awindmill. A plurality of such oriented first imagers ensures coverage ofall approaches to the building, airstrip/airfield or wind-turbine.Alternatively, a plurality of systems may be used to ensure desiredhemispherical coverage.

A moving object may be continuously identified for object movement froma first imager field of view to a spatially adjacent second first imagerfield of view, including for another first imager that is itself part ofthe system or part of a distinct second system.

As desired, the imagers may image a field of view in the visiblespectrum and/or the non-visible spectrum. For example, imaging of aninfra-red emission from the field of view is useful for detection ofliving animals of a different temperature than the surrounding airspace.Accordingly, the first imager, the second imager, or both the first andthe second imagers may be configured to detect a wavelength rangecorresponding to light in the visible or infra-red spectrum. Such awavelength range is in the infra-red is useful for identification inlow-light (e.g., night) or adverse weather conditions, or any conditionswhere color/visibility is not distinguishable.

Any of the avian detection systems may be configured to simultaneouslyidentify a plurality of moving objects and, as desired, determinethreshold identification attribute(s) and avian identificationparameters, and probabilities associated therewith.

One application of any of the avian detection systems and methodsdescribed herein is with a wind turbine and that is used to decreaseincidence of avian kills by a wind turbine, including for a specificavian species of interest that may include a raptor, or a golden eagle.

A plurality of avian detection systems may be connected to a windturbine in distinct alignment directions to provide said substantiallycomplete hemispherical coverage of said airspace surrounding the windturbine. For example, one of the first imagers may be oriented in anupward direction to cover a region of airspace above the wind turbine,whereas other imagers provide airspace coverage closer to the ground ina full 360° coverage orientation. Alternatively, the systems may bestand-alone and spatially separated from the wind turbines, such asstrategically positioned around and within an area to-be-monitored,including around a perimeter footprint of a wind-turbine or a windfarmcomprising a plurality of spatially-separated wind-turbines. In thismanner, a significant reduction in the total number of systems may berealized as there may be substantially less than a one system to onewind-turbine ratio needed to achieve adequate and reliable coverage.

Any of the systems and methods provided herein may further comprise acontroller operably connected to the processor to provide an actionimplementation. Examples of action implementation include those selectedfrom the group consisting of an alarm, an alert to an operator, a count,an active avoidance measure, or a decrease or stop to a wind turbineblade speed when the avian detection system identifies a flying avianthat is a threatened or an endangered species having a predictedtrajectory in a wind turbine surrounding airspace that will otherwiselikely result in wind turbine blade impact. As desired, for windfarmapplications, this slowing or stopping of blade speed can be for subsetof wind-turbines in the windfarm identified as being at high risk of anendangered avian turbine strike.

Another application of the avian detection systems and methods providedherein include for counting a number of flying avians and/or species ofinterest identification within the airspace surrounding an aviandetection system over a time period. This can assist with environmentalimpact statements, risk assessment and management.

The avian detection systems and methods herein are compatible withstationary applications or moving applications. For example, stationaryapplications include simple bird count surveys at a fixed location.Moving applications include those where even larger regions are to beexamined, in which case the systems can be mounted to a moving vehicle,including a land-based, sea-based, or airborne vehicle.

The systems are compatible with any kind of positioners. For example,the positioner can comprise a motorized pan and tilt head connected tothe second imager for moving an alignment direction of the second imagerbased on an output from the first imager

The first imager, the second imager, or both the first and secondimagers may be cameras, having lenses and sensors. Exemplary camerasinclude cameras having CCD or CMOS sensors.

Any of the systems provided herein may be used with a second imager thatis a stereo imager to determine distance and optionally trajectory ofmoving objects. The avian detection system for detecting a flying avianin an airspace may comprise a first imager having a wide field of viewfor detecting a moving object; a stereo imager comprising a pair ofimagers each independently having a high zoom; a positioner operablyconnected to the stereo imager for positioning said stereo imager toimage said moving object detected by the first imager; and a processoroperably connected to receive image data from said first imager, saidstereo imager, or both and to determine a position and trajectory ofsaid moving object, thereby identifying a moving object that is a flyingavian based on image data from the first imager, the second imager, orboth the first and second imager.

The avian detection system may provide substantially completehemispherical coverage of airspace surrounding the avian detectionsystem. For example, the avian detection system may comprise a pluralityof first imagers and a plurality of stereo imagers, wherein one or moreof the imagers are aligned in distinct alignment directions to providethe substantially complete hemispherical coverage of airspacesurrounding the avian detection system. For example, the first imagersmay be fixably positioned and the second imagers positionable with acontrolled alignment direction, including with a pan and tilt, toprovide coverage over a large field of view without sacrificingresolution.

Any of the avian detection systems may have a processor that iswirelessly connected to the imagers or a processor that is hard wired toobtain image data output from the first imager, the second imager, orthe stereo imager.

Also provided herein are methods of detecting a flying avian speciesimplemented by any of the systems disclosed herein.

Also provided herein are methods of detecting a flying avian in anairspace. The method may comprise the steps of: imaging the airspacesurrounding an imaging system; obtaining one or more thresholdidentification attributes for an output of a subset of pixels from theimaging step; analyzing the one or more threshold identificationattributes to identify a moving object of interest; obtaining one ormore avian identification parameters for the moving object of interest;comparing the one or more avian identification parameters to acorresponding one or more reference avian identification parameters toidentify a flying avian; and implementing an action implementation forthe flying avian; wherein the method detects the flying avian within theairspace having a volume equivalent to an average-equivalenthemispherical airspace with an average radius selected from a range thatis greater than or equal to 0.5 km and less than or equal to 1.2 km, orany subranges thereof.

In an aspect, the imaging step comprises identifying an output of asubset of pixels, such as an output that is an array of light intensityvalues.

The imaging step may comprise obtaining a wide field of view with afirst imager and optically zooming and/or focusing in on the movingobject of interest with a second imager, wherein the second imager isused to determine a position of the moving object of interest from theimaging system. The position may also be determined relative to anotherpoint fixed relative to the imaging system. For example, a ground basedimager that is at a distance from a wind turbine may be used todetermine an avian position relative to the wind turbine, therebyproviding a distance from the wind turbine. Similarly, positions anddistances from other objects may be determined, including an airplane, arunway, a building, a power-line, or any other structure.

The method may further comprise classifying a species for the flyingavian of interest. For example, the output of the subset of pixelscorresponding to a flying avian may be further analyzed with the avianidentification parameter to determine whether the flying aviancorresponds to a particular species. The particular species is alsoreferred herein generally as a “species of interest” and may correspondto a raptor or other avian of interest, depending on the application ofinterest.

The imaging step may further comprise obtaining a plurality of images atdifferent times and determining a trajectory of the output of the subsetof pixels.

Any of the systems and devices provided herein may determine thedistance of the moving object using a second imager that is a stereoimager that is positioned to image the moving object. In this manner,objects that may be large but positioned far away are positionallydistinguished from smaller objects that are located closer to thesystem.

Any of the classifying steps and/or identifying steps may comprise apattern recognition algorithm.

As used herein, the one or more threshold identification attributes maybe selected from the group consisting of distance, trajectory, boundaryparameter, boundary shape, edge boundary characteristic, pixel spacing,pixel intensity, pixel color, intensity gradient, time evolutionparameter, and any combination thereof.

The one or more threshold identification attributes may be a boundaryparameter. Accordingly, any of the methods provided herein may furthercomprise the step of comparing the boundary parameter to an edgeboundary signature characteristic of a flying avian. Examples of edgeboundary signatures characteristic of a flying avian may include shapes,colors, intensity, and relative distributions thereof. For example, foran avian that is a bird, specific shapes of wingtips, body, head, tailfeathers may provide edge boundary signature characteristics useful tocompare against the boundary parameter obtained from the output of thesubset of pixels.

Similarly, a boundary parameter may be used to determine if the movingobject that is related to the output of the subset of pixels correspondsto an artificially-constructed object, such as an airplane. This may beaccomplished by identifying a moving object as corresponding to anartificially-constructed object by identifying at least a portion of theboundary parameter as having a shape indicative of anartificially-constructed object, including an edge straightnessparameter indicative of the artificially constructed object. The edgestraightness parameter may quantify how straight a portion of theboundary is or, similarly, the smoothness of a portion of the boundary.Avians that are birds or bats typically do not have continuously highlystraight or smooth boundary edges.

The one or more avian identification parameter may be selected from thegroup consisting of size, speed, wing span, wing shape, color, boundaryshape, geometry, light intensity, flight trajectory, posture,temperature or a heat signature.

Any of the methods may further comprise the step of obtaining apredictive trajectory of the flying avian, such as based on priordetermined trajectories. For example, an avian that is soaring in upwardcircles may be predicted herein to have a similar continuing trajectory.Alternatively, an avian that is in a dive may be predicted to continuethe dive to a certain elevation followed by an abrupt pull out of thedive.

The to be detected flying avian may be a threatened species, anendangered species, or a migratory bird. For example, the threatened orendangered species is a raptor. In this manner, the systems and methodsprovided herein are readily adapted for the windfarm location, asdifferent geographic locations have political and ecological conditionsthat, in turn, affect which avians are of interest. The systems andmethods may be adapted by accordingly revising and updating the relevantreference values in a pattern recognition algorithm and avianidentification parameter. For example, a migratory sea bird may have adifferent appearance, size and flight characteristics than a goldeneagle. A system in a sea-bird detection application, therefore, may beaccordingly tailored for detection of the sea bird, whereas a goldeneagle detection application tailored for the golden eagle.

Any of the methods provided herein may utilize a comparing step thatcomprises a pattern recognition algorithm to facilitate processing andidentification.

The methods and systems provided herein represent a substantialimprovement of the art, characterized as having extremely highreliability rates of detection even over large distances. In an aspect,the system may be described in terms of a detection sensitivity that isgreater than 96% and a false positive detection that is less than 5% fora threatened species, endangered species, or a species of interest forthe airspace up to a maximum distance from the imaging system that isgreater than 0.6 km and less than 1.2 km. The effect of such rates isthat few, if any, species of interest are missed and there is little, ifany, over-detection by incorrectly assigning a species identification tothe incorrect avian. The systems provided herein, therefore, have anumber of important applications, including for a wind turbine.

Any of the systems and methods may be used with a wind turbine. Themethod may further comprise the steps of decreasing a blade wind turbinespeed or stopping movement of the blade turbine to minimize or avoidrisk of blade strike by the flying avian having the predictivetrajectory that would otherwise likely result in blade strike of theavian or that may be within an actionable interior airspace that iswithin the surrounding airspace.

In an aspect, the avian is a species that is a threatened or endangeredspecies, including those that are defined under the Endangered SpeciesAct of U.S. law for U.S.-based applications, or any species identifiedby foreign government agencies, or international treaty. In an aspect,the avian is a golden eagle.

An advantage of the methods and devices herein in a wind-energyapplication is that characterization of avian species assists withmaximizing wind turbine output by avoiding decreasing or stopping windturbine blade speed for an avian identified as not corresponding to thespecies of interest. In an aspect, the blade wind turbine speed is notactively decreased for an avian species that is identified as not anavian species of interest, thereby maximizing wind turbine efficiency.

The implementing an action step may comprise one or more of: providingan alert to a person; emitting an alarm; triggering a count event;triggering a deterrent to encourage movement of the flying avian out ofthe surrounding the first imager; recording an image or video of theavian flying through the airspace surrounding the first imager; ordecreasing or stopping a wind turbine blade speed.

The method may further comprise the step of defining an actionimplementation airspace having an average action distance that is lessthan the average-equivalent radius of the substantially hemisphericalairspace surrounding the imaging system, wherein the actionimplementation is implemented for a flying avian that is either withinthe substantially hemispherical airspace and having a trajectory towardthe action implementation airspace; or within the action implementationairspace. This aspect may be particularly useful in wind blade strikeavoidance where the flying avian is tracked in the airspace but noaffirmative countermeasure to avoid or minimize blade strike isundertaken until the flying avian is within a “danger” zone or appearsheaded to the danger zone. This danger zone may be referred herein as anaction implementation airspace that is less than the surroundingairspace, such as being similarly hemispherical but with a radius thatmay be less than 70%, less than 50% or less than 30% of the maximumdetection distance, such as for a maximum detection distance of betweenabout 600 m and 1.2 km. The higher the velocity of the bird, the largerthe danger zone range, so that appropriate countermeasures can be takenbefore a potential bird-strike. Accordingly, any of the systems andmethods may have a detection distance that is determined to ensuresufficient time for a counter-measure is available for a maximumdetermined flight speed of the avian of interest. Alternatively, thedetection distance may be actively controlled and varied depending onconditions. For example, wind speed and direction may be detected, withthe detection distance accordingly varied to increase in distance fromdirections where the avian would be wind-assisted and decrease thedistance where the avian would be flying into the wind.

Any of the methods may further comprise the step of turbine masking foran image of a flying avian in an optical region containing a movingturbine blade, thereby improving detection, including by the algorithmof FIG. 5 .

Also provided herein is an avian detection system for detecting a flyingavian in an airspace surrounding a wind turbine. The system may comprisea plurality of imaging systems, each imaging system comprising: a firstimager having a wide field of view for detecting a moving object; asecond imager having a high zoom, wherein the first and second imagersdetermine a position and a trajectory of a flying avian in the airspace;and a positioner operably connected to the second imager for positioningthe second imager to image the moving object detected by the firstimager. A processor is operably connected to receive image data from anyof the first imager, second imager, or both, and to identify a movingobject that is a flying avian based on the image data. There may be oneprocessor for each imaging system or a single processor that is operablyconnected to each of the imaging systems. Each of the plurality ofimaging systems is positioned relative to others of the imaging systemto provide substantially complete hemispherical coverage of the airspacesurrounding the wind turbine. A controller receives output from theprocessor, the controller operably connected to the wind turbine fordecreasing or stopping wind turbine blades for a flying avian identifiedas at risk of otherwise striking a moving blade of the wind turbine.

The avian detection system may comprise at least four imaging systems,wherein: at least three of the imaging systems are mounted to a windturbine or a stand-alone support structure such as a stand-alone tower,not associated with wind generation, each of the three imaging systemsaligned in a unique horizontally defined direction to provide 360°coverage by the at least three first imagers up to a vertical distance;and at least one imaging system is mounted to a nacelle or a top surfaceof the wind turbine in a vertically defined direction to providevertical coverage by the at least fourth first imager. Together the atleast four imaging systems provide the substantially completehemispherical coverage of the airspace surrounding the wind turbine orstand-alone structure, up to a distance that is greater than or equal to600 m, including between 600 m and 1.2 km.

Any of the avian detection systems may be configured as a stand-alonesystem. For example, the stand-alone system may comprise a tower thatsupports the plurality of imaging systems; a plurality of wide field ofview systems, each comprising a pair of first imagers; one or morestereo imagers, each stereo imager comprising a pair of second imagers;wherein the imaging systems are connected to the tower at a top end by atower interface that positions the plurality of wide field of viewsystems in optical directions to provide a 360° view around the tower.

The avian detection system is compatible with any number of imagers andimaging systems, such as three wide field of view systems, eachproviding a field of view between 120° and 140° and at least one or onestereo imager. A ground enclosure may be provided containing ancillaryequipment electrically connected to the plurality of imaging systems bycables that run through an inner passage within the tower. In thismanner, the equipment may be reliably secured in an anti-tamper proofconfiguration, thereby minimizing risk of loss, damage or destruction. Alightning mitigation system may extend from the tower top, wherein theimaging systems are positioned so as to image airspace around the towerwithout optical obstruction by the lightning mitigation system. Forexample, for a plurality of stereo images, the lightning mitigationsystem may be positioned at an origin so that the mitigation system isnot in an optical pathway. For a single stereo imaging the system, therelative positions are selected to minimize interference and, asnecessary, a second spatially distinct stand-alone avian detectionsystem may be positioned to ensure any blind spot is imaged by thesecond avian detection system, and/or to provide desired verticalcoverage above the first avian detection system.

Any of the systems provided herein may be configured as a stand-alonesystem. “Stand-alone” refers to the system that is independent of anyother structure. This is in contrast to systems configured to attach tostructures having other function, such as wind turbines for energygeneration.

The avian detection system may further comprise a plurality of widefield of view systems, each wide field of view system comprising a pairof first imagers forming an alignment angle with respect to each otherto provide a field of view angle for each wide field of view system thatis greater than or equal to 90° and less than or equal to 180°, whereinsaid plurality of wide field of view systems in combination provides360° imaging coverage around said avian detection system; and a stereoimager comprising a pair of said second imagers. The stereo imager canrapidly be positioned with a positioner, such as a motorized pan tiltsystem, to focus on regions of interest identified by the WFOV system.

The system is compatible with a single stereo imager, whichadvantageously decreases hardware costs, as well as with a plurality ofstereo imagers. While multiple stereo imagers increases costs, they canbe beneficial for more detailed analysis and tracking, especially for ahigh number of birds present in multiple directions. In this manner,each of the wide field of view systems may be individually connected toa unique stereo imager.

Any of the systems may connect to a tower top by a tower interface. Theavian detection system may further comprise a substrate having a topsurface and a bottom surface, wherein the positioner connects the stereoimager to the substrate top surface and the wide field of view system isconnected to the substrate bottom surface.

The tower interface may further comprise a central interface portion forsupporting the stereo imager and connecting to a top portion of a tower;and outer support struts for supporting the wide field of view imagers.

Without wishing to be bound by any particular theory, there may bediscussion herein of beliefs or understandings of underlying principlesrelating to the devices and methods disclosed herein. It is recognizedthat regardless of the ultimate correctness of any mechanisticexplanation or hypothesis, an embodiment of the invention cannonetheless be operative and useful.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 : Process flow diagram of a method of identifying an avian withinan airspace and action implementation.

FIG. 2 . Schematic side-view of a stand-alone avian detection systemthat provides hemispherical coverage (A) and substantially hemisphericalcoverage (B), with the bottom panel of B illustrating a central deadspot region that may be imaged by a second system to provide thehemispherical coverage illustrated in A.

FIG. 3 . Schematic side-view (A) and top-view (B) of a plurality ofavian detection systems mounted to an object to obtain hemisphericalcoverage around the object. Each system is characterized as providingcoverage over a defined air-space region.

FIG. 4 : Process flow diagram of an algorithm used to detect andidentify a flying avian in an airspace surrounding the imaging system.

FIG. 5 : Process flow diagram of a turbine masking algorithm for aviandetection with an intervening wind turbine.

FIG. 6 : Schematic illustration of an avian detection system mounted toa stand-alone tower.

FIG. 7 : Schematic illustration of an imaging tower with an aviandetection system portion supported thereby, with cabling and groundenclosure to facilitate a self-contained and stand-alone system.

FIG. 8A is a schematic illustration of three sets of systems, eachcomprising a pair of wide field of view sensors to form a WFOV systemand high-resolution sensors to form a stereo image sensor. An ionizingsystem ionizes air and reduces lightning strike risk. FIG. 8B is aclose-up view of one of the systems of FIG. 8A, also referred herein asan “imaging pod” having both first and second imager systems, with eachsystem formed from two imagers.

FIG. 9 is a schematic illustration of three pairs of wide field of viewsensors, with one pair of high resolution stereo sensors forming astereo imager system connected thereto with a pan-and-tilt.

FIG. 10A illustrates the system of FIG. 9 supported to a top of a towerwith an ionizer for minimizing lightning strike risk. FIG. 10Billustrates a tower interface ready to connect to tower top and receiveand support the various imager systems.

FIG. 11 is a schematic illustration of the stereo vision system,comprising a pair of high resolution sensors. A shows a perspectiveview, with the two sensing ends of the high resolution systems visibleon the face, and protected from the elements by an overhanging cover. Bis a side view. C is a bottom view. D is a perspective bottom view.

FIG. 12 is a schematic illustration of the wide field of view sensors,with A illustrating the outer cover that surrounds the sensors. B is aview of the pair of sensors positioned in the cover, such as at arelative angle with respect to each other of about 60°, with each sensorproviding about 60° field of view coverage, and at least about 120°coverage. C is a view of the system of A flipped over to betterillustrate the positions of the sensors within the outer cover.

FIG. 13 is a schematic illustration of the wide field of view sensors ofFIG. 12 , from an A bottom perspective view; a B top view; and a C sideview.

FIG. 14 illustrates a ground enclosure or self-contained equipment (toppanel) and corresponding components contained therein (bottom panel).

FIG. 15 is an overall illustration of the various components and groundequipment useful with the systems provided herein. For clarity, the topof the tower and corresponding imaging systems connected thereto is notshown. The top panel is a perspective view and the bottom panel a sideview to illustrate the electrical ground to mitigate lightning strikeand corresponding damage to the imaging system and related electronics.

FIGS. 16A-16C are photographs of an avian with different posture ororientation. A ratio of the wingspan (w) and height (h) may be used tobetter estimate wingspan. This pose-estimation may be used to providemore accurate estimation of true wingspan, as summarized in thecorresponding plot of FIG. 16D.

FIG. 17 is a schematic illustration of an observation tower supportingan avian detection system positioned within a wind-farm footprint. A120° field-of-view provides coverage of a plurality of distinct windturbines, including up to a distance of 1 km from the tower. Completecoverage is achieved by incorporating additional wide field of view(WFOV) sensors to provide 360° coverage, as desired. Additional systemsmay be positioned to cover other wind turbines. In this manner,significant reduction in hardware and costs are realized as fewersystems are required for complete coverage. This also illustrates thescalability of the instant systems, with any size of airspace that canbe monitored with addition of more systems.

FIGS. 18-19 are photographs of a raptor before (FIG. 18 ) and after(FIG. 19 ) stereo measurement. The raptor distance is about 1 km fromthe system

FIG. 20 is a photograph of a system tracking a raptor. The top sensorimage is from the high-resolution sensor and the bottom sensor imagefrom the WFOV sensor, with the ground and wind turbines visible.

FIG. 21 is a series of sensor images labeled (i)-(v) of a pair of avianstracked in the WFOV, with a first avian maintaining a gliding heightabove the ground and wind turbines. A second avian approaches the groundas shown in panel (ii) and flies behind a turbine (iii) and closer alongthe ground (iv) before increasing height from the ground (v). Each avianis successfully tracked.

DETAILED DESCRIPTION OF THE INVENTION

In general, the terms and phrases used herein have their art-recognizedmeaning, which can be found by reference to standard texts, journalreferences and contexts known to those skilled in the art. The followingdefinitions are provided to clarify their specific use in the context ofthe invention.

“Avian” is used broadly herein to refer to a flying animal. Accordingly,the term encompasses birds, bats and insects. Particularly relevantavians for the methods and systems provided herein are flying animalsthat are endangered, threatened, or otherwise of commercial orenvironmental interest. In an aspect, the avian is a bird or a bat. Inan aspect, the avian is an avian bird species of interest such as araptor and/or eagle species that may be endangered or threatened. In anaspect, the avian species is a golden eagle.

“Airspace” is used herein to refer to a volume of space that surroundsthe detection system. To clarify that the systems provided herein areconfigured to detect a flying avian in any observable direction from thesystem, the airspace is generally referred herein as hemispherical. Inthis context, “hemispherical airspace” refers to an all-directionalcoverage from a point of origin corresponding to an imager of thedetection system. Accordingly, the imager(s) of the systems providedherein permit azimuth angle coverage 0°≤φ≤360° (0°≤φ≤2π rad) (FIG. 3B);and inclination (or elevation) angle coverage 0°≤θ≤180° (0°≤φ≤π rad)(FIG. 2A) for a defined detection of interest distance up to a maximumdistance corresponding to r (215 of FIG. 2A). The actual airspacecorresponding to such a hemispherical detection may itself benon-hemispherical, reflecting real-world conditions where there may beobstructions to line of sight from an imager (FIG. 2B, top panel), deadspace immediately above certain systems (FIG. 2B, bottom panel) or aplurality of spatially separated imagers (FIG. 3 ) that provide“bulging” hemispheres and the like.

“Substantially hemispherical” refers to a volume of airspace defined interms of a center of origin and extending out a user-selected distance,but that may deviate from a true hemisphere volume, defined as ⅔πr³, fora half-hemisphere with the ground bisecting the hemisphere, where r isthe average maximum detection distance from the center of origin, suchas corresponding to the position of the avian detection system, asillustrated in FIG. 2B. In aspects where the volume is to bequantitatively expressed, the deviation may be expressed as less thanabout 20%, less than 10% or less than 5% of a hemisphere volume for acorresponding “average” distance, r, for the system. In addition, for aplurality of detection systems that are spatially positioned atdifferent positions with respect to each other, the hemisphere may bulgeoutwards or inwards in certain locations while each individual systemmay have a generally half-hemisphere coverage shape. Irrespective of anysuch deviations, the systems and methods provided herein have a commonfeature of reliably detecting flying objects at a distance, includingrelatively large distances of up to about 600 m to 1.2 km, over visuallyobservable directions defined as 0°≤φ≤360°, 0°≤θ≤180°, and 0°≤θ≤120°. Asdesired, certain directions may have a larger detection distance thanother directions. For example, depending on wind direction (andconsequently blade face direction), a certain detection directioncorresponding to the wind direction and perpendicular to the blade facemay be extended. This can accommodate increased flying avian groundspeed for a flying avian direction that is aligned with the wind.Accordingly, the airspace generally described as substantiallyhemispherical may include more elliptically-shaped airspaces with amajor axis aligned with the wind direction. The major axis may begreater than 20%, 40% or 60% than the minor axis.

“Substantially complete hemispherical coverage” refers to airspacecoverage, with respect to an origin corresponding to an imager(s) orsensor(s) that essentially covers all possible directions of approach ofa flying animal toward the imager. In other words, provided is acomplete line-of-sight coverage. Accordingly, as necessary additionalsystems may be utilized to cover any dead-space regions that do not havegood line-of-sight coverage from a first system.

“Imager” refers to any devices that obtain images of airspacesurrounding the system. The imager may comprise a camera, includingassociated optical components, such as lenses, sensors, filters,diffusers, and the like. Exemplary cameras include cameras having CCD orCMOS sensors. The image may be of visible light or non-visible light.For applications where the avian of interest tends to fly in thedaylight and in non-storm/fog conditions, a visible light camera may beused. In contrast, for nocturnal avians that tend to fly in low-lightconditions, such as bats, an infra-red camera that captures infra-redimages may be used. To provide 24-hour coverage, both visible light andinfra-red cameras may be used. “Sensor” is used herein as generallysynonymous to imager, and reflects the systems can track moving objectswithout having to actually display an image to a user, but instead maybe implemented with software to automatically track and take appropriateaction depending on the tracked moving object.

“Positioner” is used broadly herein to refer to the ability to positionthe second imager to focus tightly, such as by zooming and/or focusing,on a moving object that may have been identified by the first imager.Accordingly, a positioner may be a motorized driver that actively alignsthe second imager to a desired viewing direction. The positioner maycontinuously align the second imager with time so that a moving objectis constantly zoomed in on and in focus with the moving object. Thepositioner may be a motorized pan and tilt to provide full spatialorientation of the second imager. Alternatively or in addition, thepositioner may be implemented with a second imager that is functionallya digital zoom. In this aspect, the positioner may be functionallyimplemented within software to provide digital zoom of the output of thesubset of pixels from the first imager.

“Processor” is used broadly herein and may include hardware, such ascomputers and computer-implemented processes. Examples of computerresources useful in the present systems and methods includemicrocomputers, such as a personal computer, multiprocessor computers,work station computers, computer clusters and grid computing cluster orsuitable equivalents thereof. Preferably, algorithms and softwareprovided herein are embedded in or recorded on any computer readablemedium, such as a computer compact disc, floppy disc or magnetic tape ormay be in the form of a hard disk or memory chip, such as random accessmemory or read only memory.

“Wide field of view” (WFOV) refers to an imager, generally a “firstimager” that can image at least a substantial portion of the surroundingairspace. For example, a fish-eye lens may be used to image asubstantially hemispherical airspace. Examples include imagers having amatched resolution to the WFOV area, such as resolution of about4608×3280 (15.1 Mpixels) to provide a desired full field of view that isgreater than or equal to 120°, such as about 130° FOV, when paired withan appropriate aligned second WFOV imager. For example, each WFOV imagermay be selected to cover about 65° at about 800 m, so that combining apair of such WFOV imagers provides 130° FOV and, therefore, canaccommodate some lens distortion. The WFOV imagers may provideindependent inspection areas or may be stitched together. Imagersconfigured to provide independent inspection areas can, depending on theimage processing and analysis, be faster. As desired, the WFOV imagersmay be periodically calibrated to ensure accuracy. A Kalman filter maybe employed for predictive tracking behavior. A configurable autoexposure and other settings may be used to improve accuracy.

“High zoom” refers to an imager, generally a second imager or a stereoimager, configured to tightly focus on a potential or detected movingobject identified by the first wide field of view imager. The high zoommay have a variable focal distance that is capable of achieving largefocal length factors. In embodiments, the high zoom provides a highdegree of image magnification, such as to access optical parameters ofinterest to assist with image classification, such as identification ofa moving avian and upon such identification classifying oridentification of a specific species or type of avian. The high zoom mayalso be referred to as having a “high resolution” tailored to the avianof interest that is being tracked, such as about 1280×960 resolution(1.2 Mega Pixel) to 1920×1440 resolution (2.8 Mega Pixel), and can betailored to the operating conditions and avian of interestcharacteristics (e.g., size). In this manner, a sensor or imager andcorresponding optical components are matched to generate an ideal pixelsize in a CCD sensor space for optimized image quality in a confinedfield of view. Attendant optical components, such as high qualityoptical filters may be used. Examples of optical components used withthe imagers include Tamron or Nikon 300 mm varifocal lenses. The highzoom may correspond to a stereo camera.

“Detection efficiency” is an indication of the reliability of the systemin detecting an avian species of interest that enters the airspace andcan be expressed as the number of avians of interest detected by thesystem divided by the total number of avians of interest that enter theairspace. The systems and methods provided herein may be described ashaving a high detection efficiency, such as greater than 95%, greaterthan 99%, or greater than 99.9% when active. Similarly, “false positive”refers to the number of avians identified as a species of interest thatdo not actually belong to the species of interest. This number isdesirably small as otherwise there may be wasted resources associatedwith an action implementation for an avian erroneously identified as anavian species of interest. In an aspect, the percentage of falsepositives is less than 5%, less than 1% or less than 0.1%.

“Output of a subset of pixels” refers to a region of the digital imagecaptured by an imager that may correspond to a moving region ofinterest. That moving region of interest is defined by a subset ofpixels, wherein each pixel is associated with an intensity value. Thesubset of pixels may be described as being formed from neighboringpixels. “Neighboring pixels” refers to pixels that are within auser-defined pixel number of each other. In an aspect, neighboringpixels refers to pixels within about ten pixels of each other. Theoutput may also comprise tightly clustered pixels that are described asbeing directly adjacent to each other. Of course, the subset may includea combination of neighboring and adjacent pixels.

“Time varying output” refers to the subset of pixels having an outputthat changes with time. This change may be associated with motion ormovement of the subset of pixels and can be a useful parameter in imagecharacterization and identification.

“Threshold identification attributes” refers to an initialcharacterization of a subset of pixels as corresponding to a movingobject and upon which further analysis may be conducted. Examplesinclude object distance, position, trajectory, boundary shape, size,color, and/or heat signature. Pixels and corresponding objects that tendto fail one or more threshold identification attributes are likely not aflying avian and so may be disregarded from further analysis or ignored.

“Edge detection” refers to systems, algorithms and processes thatidentify points or pixels in a digital image whose intensity orbrightness changes, such as by a discontinuous change in lightintensity. The various points or pixels having such sharp imagebrightness change are accordingly organized into line segments referredherein as an edge. Edge detection is useful herein in various imageprocesses including detection of a moving object and classification ofsuch objects. In an embodiment, the edge detection is by determining agradient of intensity and classifying an object as having an edge for agradient that exceeds a user-selected gradient of intensity. Such edgedetection may be a useful part of obtaining a threshold identificationattribute for the subset of pixels.

“Boundary parameter” refers to a parameter that is reflective of atleast a portion of or all the edge of the subset of pixels. Examples ofboundary parameter include edge shapes, total perimeter, interior area,intensity, and localized variations thereof. Particularly usefulboundary parameters include those that may be compared against an edgeboundary signature that is characteristic of a flying avian. Forexample, flying avians may have unique wing shapes, motion, curvaturesand surface ruffling or roughness, with distinct front ends (e.g., head,beak, etc.) and back ends (e.g., tail feathers). Any such aspect that ischaracteristic of a flying avian is generally referred herein as to an“edge boundary signature characteristic of a flying avian” and may beutilized herein in a preliminary analysis of the subset of pixels todetermine if further analysis is warranted.

“Reference values” refers to any parameter associated with a flyingavian, including an avian of interest. The reference values may beobtained from empirical evidence, such as avian shapes, color, sizes,flying pattern, thermal signature, etc. Alternatively, the referencevalues may themselves be machine generated by visualizing a known avianand generating the parameters under real-world flying conditions. Inthis aspect, a trained avian such as a raptor can be used for imageacquisition and according edge boundary signature determination that ischaracteristic of the trained avian. In an aspect, the trained avian isa golden eagle. As desired, any such reference values may be stored in areference image database for use by any of the systems and methodsprovided herein.

“Avian identification parameter” refers to any parameter useful fordetermining whether a subset of pixels corresponds to a specific avian.Examples include size, speed, wing span, wing shape, color, boundaryshape, geometry, light intensity, and flight trajectory. For an imagethat is an infra-red image, the parameter may correspond to temperatureor a heat signature. Conceptually, the avian identification parametermay be similar to the edge boundary signature characteristic of a flyingavian, but may be tailored toward a specific avian to provide enhancedspecies identification. The added computational resources and time forobtaining reliable avian identification parameters and using them forspecies identification makes this aspect useful only for those movingobjects that have been defined as potentially avian from the initialboundary-related analysis.

Any of the systems described herein may have automated start and stoprecording, such as based on weather conditions, daylight conditionsand/or moving object detection. This facilitates raw recording from allimagers according to one or more configurable settings, such as factorypre-sets or user-selected setting. Examples of such a setting can beautomating recording if a low or high priority object is tracked forlonger than 0.5 seconds. Accordingly, the system may be run 24/7, withcertain systems that are set to not record data at night forapplications where night-time tracking is not desired. Alternatively,the system may be set to forced record if a noteworthy event has orrecently occurred, such as a bird strike on a turbine.

The systems may have a custom logging script to provide pan tilt errorassessment and appropriate corrections. For example, as wind turbinesare generally located in exposed high-wind locations, high wind gustsmay cause a pan tilt slip, and the error correction may reset the pantilt to a desired position. As desired, pixel location may be convertedand expressed in terms of degree relative to an origin, such as locationof the imager. Manual control may be provided, such as user-control ofthe pan tilt system for a user-override of the second imager. Forexample, a user may manually click a location on a WFOV image so thatthe high resolution imagers automatically zoom on that location.

Any of the systems may include an auto exposure to optimize visibilitythrough the day as lighting conditions vary, such as by varying one ormore of exposure, gain and/or image quality. For example, during eveningand early morning the high resolution imagers may log exposure time,with a maximum exposure time so as to not blur a bird moving at highspeeds while not adversely impacting image quality required to make ahigh-accuracy avian characterization. Gain may be dynamically adjusteddepending on the time of day, as toward evening light will keep gettingdimmer.

Example 1: Detection Methods

Referring to FIG. 1 , an overall flow process of a general strategy fordetecting a flying avian in an airspace is by imaging at least a portionof or all the airspace surrounding the system with a wide field of viewimager 100. For example, in an unobstructed airspace a single wide fieldof view imager having a wide-angle optical system (see, e.g., U.S. Pat.No. 8,284,258) may image an entire hemisphere. Alternatively, forimagers that cover less than an entire hemisphere of surroundingairspace, a plurality of imagers may be used to provide completehemispherical coverage. This may be particularly useful in situationswhere there is a physical obstruction, such as with a building, tower,tree, wind turbine nacelle, or the like. In such situations, more thanone imager may be strategically located to provide multiple fields ofview, that when combined, provide complete or substantially completehemispherical coverage.

A wide field of view imager or imagers are useful for identifying amoving region of interest 110, which may be described in terms of anoutput of a subset of pixels of the imaged airspace. The moving regionof interest may be detected or identified by comparing images of a fieldof view at different time points and detecting changes in the image,such as would occur with a moving object. One example of a technique isby determining changes in pixel intensity and identifying such a changein pixel intensity as a region of interest. Tracking movement of such achange in pixel intensity over time provides a moving region ofinterest. In an aspect, a plurality of moving regions of interest isidentified, with each region individually tracked.

For a moving region of interest, distance of the moving region ofinterest relative from a user-selected geographical location may beobtained 120. For example, a second imager having a high zoom forfocusing tightly on the region of interest, may provide distanceinformation. For example, the level of zoom magnification correspondingto a highly focused image may provide information about the distance ofthe moving region of interest. Another example is a stereo imager thatobtains a stereo image of the moving region of interest to measuredistance from the moving region of interest and the stereo imager (see,e.g., U.S. Pat. No. 6,411,327; Mahammed et al. “Object DistanceMeasurement by Stereo VISION.” IJSAIT 2(2): 5-8 (2013)). Other examplesof a second imager include two camera systems, such as two chargecoupled device (CCD) cameras. The methods and systems provided hereinare compatible with a range of imagers and methods that provide distanceinformation of an object being imaged. In this manner, distance of themoving region of interest from the systems provided herein is obtained.If the moving region of interest is outside a user-selected region, theregion of interest may be characterized as outside the user-selectedairspace with no further action taken 130. Alternatively, the movingregion of interest may be periodically or continuously monitored toensure it does not move within a distance that is within theuser-defined airspace. Depending on the application of interest, theuser-defined airspace is selected by a distance range. As discussed, theuser-selected distance range that defines the airspace of interest canbe defined as between about 600 m to 1.2 km, and any sub-ranges thereof.Of course, other distance ranges are compatible with the devices andmethods provided herein. For example, if a plurality of systems isprovided to ensure substantially complete hemispherical coverage, thedistances (and/or trajectories) for an individual system may bedifferent so as to achieve a “final” airspace coverage around allpossible approaches, thereby providing substantially hemisphericalcoverage with respect to a geographical point of origin.

For a moving region of interest, trajectory of the moving region ofinterest relative to a user-selected geographical location may beobtained 120. For example, the trajectory may be determined orcharacterized from a plurality of images over time to provide an averagetrajectory. Similarly, an anticipated or predicted trajectory may bedetermined based on the past trajectory. The predicted trajectory may beexpressed in terms of a probable trajectory track, such as with outertrajectory confidence limits that define a percentage likelihood, suchas a 50% likelihood, a 75% likelihood, a 90% likelihood or a 95%likelihood. Higher outer percentage limits increase the trajectory outerconfidence limits. Any of the methods and systems provided herein mayoptionally implement an action based on a user-selected trajectoryconfidence limit (e.g., 50%, 75%, 90% or 95%) that intersects with ageographical point of interest. Conceptually, referring to FIG. 2A, eachtrajectory 240 from object 230 will have an envelope of trajectory outerlimits along with an “average” predicted trajectory depicted by an arrow240. This provides additional robustness and certainty to the aviationdetection systems provided herein, particularly for applicationsrequiring a stop-type action to a moving wind-turbine blade centeredaround 200 of FIG. 2A, such as within an actionable distance defined bydashed arrow 225 and corresponding dashed hemisphere surface 220.

In this manner, step 120 may be as simple as determining whether or nota moving region of interest is at a sufficiently “close” distance.Alternatively, the step 120 may be more complex by also considering atrajectory (see, e.g., elements 230 (object of interest distance) and240 (object of interest trajectory) of FIG. 2A). Moving objects within auser selected distance (see, e.g., distance 215 of FIG. 2A) may becarefully monitored with trajectory 240 plotted: with no action requiredfor moving objects having a trajectory away from 200, (see, e.g., 231)and action required for moving objects having a trajectory toward 200(see, e.g., 232). In contrast, moving regions of interest determined tobe outside the airspace defined by distance 215 (see, e.g., 230 of FIG.2A) may be further monitored or ignored, until such time as the movingregion of interest enters the airspace (see, e.g., 233).

For moving regions of interest that are within user-defined distancesand optionally having a trajectory of interest, the moving region ofinterest is then examined in step 140 and identified as not an avian 150or an avian 160. For example, if the moving region of interest is apiece of blowing debris, such as a leaf, a piece of refuse, or the like,the moving region of interest may be disregarded. Alternatively, if themoving region of interest is identified as an avian, optionally the nextstep is to characterize the avian, such as by determining the avianspecies or whether or not the avian corresponds to an avian species ofinterest 160.

The step of identifying a moving region of interest, as well assubsequent steps such as whether the region is an avian or an avian ofinterest, is compatible with any number of processes known in the artthat provide rapid, reliable and robust image analysis, identificationand/or recognition. For example, edge detection may be used with any ofthe methods and systems provided herein. Although many criteria andparameters are available for pattern recognition, one useful aspect isthe straightness of the edge. An extremely straight edge or uniformlycurving edge is indicative of an artificial object, such as an airplane,helicopter, hot-air balloon or other man-made object. Flying animals, incontrast, do not typically have such edges, but instead are feathered orotherwise not so straight or smooth. Accordingly, such a patternrecognition may be used to determine the moving region of interest isnot an avian 150. If the avian is not an avian of interest 170, theavian (and the moving region of interest corresponding thereto) may beignored.

In contrast, other edges may be highly indicative of an avian, such astail feathers, wing feathers, wing tip, beaks, and the like. As desired,multiple such parameters may be used to further improve patternrecognition and avian classification. Similarly, for other animalspecies such as bats or insects, the edges associated with those animalspecies may be utilized in the one or more pattern recognitionalgorithms.

Accordingly, one unique aspect of the systems and methods providedherein is the reliable and efficient manner in which moving regions ofinterest (corresponding to subset of pixels) may be subsequently ignored(at least temporarily), including: outside a user-defined distance ortrajectory or combination thereof; a moving region of interest that isnot an avian; or a moving region of interest that is not an avian ofinterest. All these aspects assist in substantially reducing the numberof false positive identifications, including to less than 10%, less than5%, or less than 1% of the total number of identifications. Such areduction in false positive is obtained without sacrificing aviandetection sensitivity, such as a sensitivity so that greater than 10%,greater than 5%, or greater than 1% of all avians of interest enteringthe defined airspace are detected.

For an avian species of interest that is within the defined airspace andoptionally headed in a trajectory defined by the user as being relevant,an action implementation 180 may be undertaken, dependent on theapplication of interest. For example, if the application is a simpleavian count system, the action implementation may correspond to anincrease in a count. If the application is an avian avoidance system,the application implementation may correspond to noise, light, or othersignal deterrent to encourage the flying avian to change flighttrajectory. If the application is with a wind turbine, the actionimplementation may correspond to decrease or stop of wind turbine bladespeed to minimize risk of bird strike and/or injury to the bird or theequipment.

For systems having a plurality of wide field of view imagers, theprocess summary of FIG. 1 may then have a plurality of such processesfeeding into a single action implementation step 180.

Example 2: Hemispherical Coverage—Single and Plurality of ImagingSystems

Referring to FIGS. 2-3 , the systems and methods provided herein providegood coverage over a well-defined airspace. FIG. 2A is a side viewschematic of an avian detection system 200 for detecting a flying avian(230 231 232 233) in an airspace 210. In this example, airspace 210 ishemispherical and is defined by a distance 215. In contrast, FIG. 2B(top panel) illustrates an embodiment where there is an obstacle 250that results in a volume of airspace that may be partially opticallyblocked 255. The obstacle may be an artificial object such as abuilding, tower or vehicle, or naturally occurring such as a hill, treeor boulder. Accordingly, the airspace may be described herein as being“substantially hemispherical” in recognition that unless the surroundingground is flat and without obstruction, deviations from truehemispherical is expected. FIG. 2B further emphasizes that the term“substantially hemispherical” refers to a system configured to image allpossible directions of approach of a flying avian toward the system 200as to approach system 200 the avian at some point must enter theoptically observable airspace. As described, a second system may belocated to provide detailed coverage of the dead space or blind spotregion 255. For another system, there may be a dead space 90 directlyabove system 200 (FIG. 2B, bottom panel). That dead space may be coveredby employing a second system positioned away from the first system sothat dead space 90 is imaged.

A flying avian has a position and a trajectory, defined by each of thefour x's in FIG. 2A and corresponding vectors 240. Certain objects maybe positioned outside the airspace of interest 210 and may bedisregarded (see, e.g., 230). Other moving objects may be within thedefined airspace (see, e.g., 231 and 232). Still other moving objectsmay move from outside the airspace of interest to inside the airspace ofinterest (see, e.g., 233) and, therefore, be subject to furtheranalysis.

Airspace surrounding the system may be defined in terms of a distance215 from the imager. For simplicity, FIG. 2A is a cross-section of ahemispherical airspace of a radius provided by distance 215. Of course,the systems and methods are not confined to true hemispherical shapeairspaces. As desired, other volumes are contemplated. For example, thevolume may bulge out or in in certain directions, depending on theapplication of interest. For avian strike applications around airportrunways or planes, the airspace volume along the direction of airtraffic may be favored, so that the hemisphere top-view cross-section isconfigured in a more ellipsoid shape to provide additional detectiondistance in the direction of airplane motion, with a relatively shorterdistance in a direction perpendicular to direction of travel.Irrespective of the specific volume shape of the airspace, a commonaspect of the systems and methods provided herein is reliable imagingand aviation detection for large airspaces in any approachabledirection, with the resultant airspace that may be described in terms ofan equivalent average hemispherical shape having an average radiusselected from a range that is greater than or equal to about 500 m andless than or equal to about 2 km. In an aspect, the average radius isbetween about 800 m and 1.1 km, such as being sufficient to take anappropriate action step when an avian is detected in the airspace.

Depending on the application of interest, avians (231 232) within theairspace may be detected, but action taken only for an avian having atrajectory that would otherwise impinge on an actionable airspace volumedefined by action distance 225 that is less than distance 215. Actiondistance 225 may be defined in terms of a percentage of distance 215,such as less than 80%, less than 60%, or less than 50%. For example, fora wind turbine application (e.g., system 200 mounted on or near a windturbine), a flying avian 231 that is simply passing through an edgeregion of the airspace may not require an action implementation as thereis a very low likelihood of a wind turbine strike. In contrast, an avian232 that is headed toward a wind turbine may require an actionimplementation, such as stopping or at least decreasing a wind turbineblade speed. The avian 232 may be tracked and if the trajectory changes,the action implementation may be stopped. Similarly, regardless of aviantrajectory, for a flying avian positioned within an action airspace(such as defined by the dashed lines of FIG. 2A), action implementationmay occur automatically with the recognition that the avian is so closeto the wind turbine, such as a wind turbine centered at a positioncorresponding to 200, that immediate action should be taken.

FIG. 3 illustrates a plurality of imaging systems 200 a 200 b 200 c 200d in a side view (FIG. 3A) and a top view (FIG. 3B) and attached to anobject 201 such as a wind turbine, a building, a tower, or the like.Similar to FIG. 2A, for simplicity the airspace surrounding the systemsis illustrated as hemispherical. Of course, particularly for themultiple imager systems, the airspace may not be truly hemispherical, aseach position of the imager having different heights and relativepositions to potentially generate bulges and/or pinches in the airspacefor a user-defined distance 215. Irrespective of any suchnon-uniformity, the airspace may still be defined in terms of ahemispherical volume with an effective volume that corresponds to thevolume of the non-hemispherical airspace, as illustrated in FIG. 3 .

FIG. 3 illustrates four imaging systems each responsible for imaging aportion of the airspace indicated by 216 a-d (FIG. 3B), therebyproviding substantially hemispherical coverage around object 201, suchas a wind turbine. FIG. 3B top view is highly schematized for clarity toillustrate the geometrical configuration that provides substantiallycomplete hemispherical coverage. Of course, there may be substantialoverlap in the wide field of view between the first imagers of each ofsystems 216 a-d, so long as the effective fields of view being imagedprovide substantially complete hemispherical coverage. With respect tothe relative positions of the imagers in each of the systems, theplurality of first imagers associated with each of 200 a-d are arrangedin distinct alignment directions indicated by 216 a-c in FIG. 3A. Eachof 200 b-d may be described as imaging up to a vertical distance 310,with the remainder of the vertically directed airspace imaged by 200 aarranged in a vertical alignment direction corresponding to thedirection of the arrow 216 a. As desired, the plurality of systems maythen be described correspondingly as that described for FIG. 2 .

Example 3: Pattern Recognition Algorithms

The systems and methods provided herein are compatible with any numberof pattern recognition algorithms known in the art, including theprocess summarized in FIG. 1 . Referring to FIG. 4 , a more detaileddescription of an algorithm is provided herein. The first imageridentifies an output of a subset of pixels from the field of view 400,such as an array of intensity values. From the output of the subset ofpixels, one or more threshold identification attributes are identified410 and used to determine whether further analysis is warranted 420. Thethreshold identification attributes may be a signature that ischaracteristic of a flying avian. The threshold identification attributemay be a characteristic of the pixel arrangement, intensity, distance,time evolution (e.g., position, distance and/or trajectory). A commonaspect of the threshold identification attribute(s) is that it providesthe capability of quickly and reliably determining whether the subset ofpixels warrants further analysis. Examples of threshold identificationattributes include position and/or trajectory, with pixels correspondingto a moving object outside a user defined airspace suggesting that nofurther analysis is required. Other examples of threshold identificationparameters include pixel spacing, pixel number, pixel movement, color,intensity gradient, edge boundary parameter including shape, timeevolution, and combinations thereof.

As desired, a plurality of threshold identification attributes may beidentified to provide added first-pass accuracy. If the one or morethreshold identification attributes indicates further analysis is notwarranted, further processing or analysis of that subset of pixels maybe avoided, as indicated by 422 (No). Otherwise, additional analysis isperformed, including identifying one or more avian identificationparameters 430 to assist in avian identification and/or avianconfirmation. Avian identification parameters can include one or more ofsize, color, color distribution, plumage, boundary shape, wing shape,speed, direction of motion, wing movement, and any other parameter knownin the art to assist with avian identification. As desired, the avianidentification parameters may be selected to correspond to a specificavian species, such as an eagle or raptor, including, for example, agolden eagle. The identified avian identification parameters from 430are compared to corresponding reference values in step 440. For an avianmatch 450 that is affirmative an action implementation step 460 mayoccur. In contrast, if there is not an avian match, further analysis ortracking of that subset of pixels may end as indicated by arrow 452.

The subset of pixels may be in a region together and described as beingneighboring pixels, adjacent pixels, or both. As indicative of a movingobject, the output may be a time-varying output, a spatial-varyingoutput, or both. For example, the output may change as the object movesto a different position in the field of view so that the subset ofpixels changes position. Similarly, as the object approaches or movesaway from the first imager, the absolute number of the subset of pixelsmay increase or decrease. Similarly, the absolute intensity values ofthe subset of pixels may increase or decrease with time or relativeorientation of the moving object with the imager(s).

For wind farm turbine applications, it is particularly important thatthe system successfully track a moving object that may be an avian, evenif there is a moving turbine blade in the field of view. Depending onthe relative positions of the imager, the moving turbine blade and theavian, the turbine blade may be relatively far or close to the avian.For example, the turbine may be between the imager and the avian or theavian may be between the imager and the wind turbine. This isparticularly relevant for applications where the imager is in astand-alone configuration mounted on a tower. Accordingly, any of thesystems provided herein may utilize a turbine masking algorithm, such asthe algorithm summarized by the flow chart of FIG. 5 . In this manner,reliable tracking is maintained even for a sensor that has both a birdand blade in the field of view.

Referring to FIG. 5 , in step 510 a frame difference is computed betweenan instant frame and last frame, with a corresponding output that is aframe difference. A threshold frame difference of step 520 outputs aframe difference element based on the frame difference of step 510. Aplurality of frame difference elements, labelled “n” provides an outputthat is an averaging element in step 530 that is subsequently used instep 540 to output a new moving average. The new moving average is inputwith a previous turbine envelope to generate a new turbine envelopeoutput in step 550. A subtraction element is created in step 560 forpositions in the vicinity of the turbine and within a certain radiusthereof. A final turbine mask is obtained in step 570 that is used toincrease tracking reliability of a moving object in the vicinity of theturbine.

Example 4: System Configuration

The detection system is designed to accommodate mild site maintenanceand service. For stand-alone systems, options include a tower that canbe tilted to the ground for servicing. Alternatively, a boom truck maylift a technician to the top of the imaging tower. Critical componentsare embedded in anti-tamper enclosures, including for the imaging tower,ground enclosures, and imaging pod.

FIGS. 6-7 illustrate systems configured for positioning on top of astand-alone tower, including a tower that may be transported and/orassembled at a desired location. In addition, the system may beintegrated directly, or indirectly, with a wind turbine, as discussed.This may be a convenient alternative to placement on a wind turbinenacelle, as installation may occur without impacting or otherwiseaffecting wind turbines either during manufacture or post-installationin the field. Referring to FIG. 6 , three WFOV imaging systems 610 arearranged to provide complete circumferential coverage around astand-alone tower 605, with each imaging system 610 comprising a pair ofWFOV first imagers 615. A second imaging system 620 comprising a highresolution camera 625, such as a stereo imaging system comprising a pairof high resolution imagers or cameras 625 provides detailed imaging andinformation for a moving object detected by the first imager(s). A pantilt 630, a type of positioner, connects the second imaging system 620to the tower and allows the second imaging system to move to focus on aregion of interest identified by any of the first imaging systems 610with the second imagers 625. The first imagers 615 of first imagingsystem 610 may be fixably positioned. In this example, there are threedistinct first imaging systems 610, with each first imaging systemcomprising a pair of WFOV sensors or first imagers 615. In this manner,complete 360° coverage by the WFOV first imagers 615 is obtained withzoom capability provided by second imaging system 620 comprising a pairof second imagers 625, with a resultant total of eight video streams. Tothe extent a vertical airspace region coverage is desired, such as inthe volume immediately above the system illustrated in FIG. 6 , a secondsystem that is positioned away from the system can provide such coveragefor this first and second imaging systems that do not provide absolutevertical coverage.

FIG. 7 illustrates a complete stand-alone avian detection system,including enclosure 650 for reliably and ruggedly enclosing sensitiveelectronic components and controller. In this illustration, the systemcomprises three imaging systems 610, each comprising a pair of firstimagers 615, with a second imaging system 620 paired to each firstimaging system, supported by tower 605. A lightning mitigation system640 reduces risk of lightning strikes to the system. A close up view ofthe avian detection system imagers is provided in FIG. 8A and of oneimaging pod comprising a first pair 615 and second pair 625 of imagersin FIG. 8B. A substrate 800 may be used to connect the positioner 630 tosubstrate top surface 810 and wide field of view system 610 to substratebottom surface 820. This increases stability of the positioner,decreasing need for calibration. In addition, such an imaging pod canhave a relatively small footprint (e.g., less than 45 cm×50 cm×45 cm(H×W×D), with a total mass of less than 22 kg. Another example of anavian detection system comprising a single high resolution stereo camerasystem 620 with three WFOV imaging systems 610 is illustrated in FIGS.9-10A. For ease of installation and servicing, the tower interface maybe modular in nature, as shown in FIG. 10B. The imaging pods may beconnected to tower 605 with a tower interface 609. The tower interface609 is configured to support three imaging pods to provide thecircumferential coverage around the tower 605 as shown in FIG. 8A. Withmodification to lightning mitigation system 640 position, the towerinterface can support a separate single stereo imager (see, e.g., FIG.10B). Cables may extend through an interior passage 606 in tower 605.The tower interface may have outer support struts 611 for supportingWFOV image systems and central interface portion 617 for connecting to asecond imager, such as a stereo camera and to facilitate placement on atop end of a tower, all while ensuring reliable connection withoutimpeding desired field of view, ease of installation, and ease of fieldmaintenance and/or replacement. In particular, individual imagingsystems of the system as a whole are readily replaced.

Lightning mitigation system 640 may be a lightning rod, or may be asystem that ionizes the air surrounding the detection system imagers.The system may electrically connect to a single earthen rod, such as achemical rod, in a 10 foot deep burial hole with access cover that isbackfilled with conductive/dissipative soil (FIG. 15 , bottom panel).The system is ideal for sand/aggregate soil having low conductivecontent.

The imaging tower 605 has an optimized height for avian detection andclassification of between about 5 m and 10 m, or about 6.3 m or 9.1 m.The tower is configured for a load rating of 113.4 kg, with a ballastedbase, such as precast cement blocks. A lift/lower mechanism may behand-cranked or motorized to facilitate transport, deployment andmaintenance.

A more detailed illustration of a stereo imager 620 is provided in FIG.11 , panels A-D. A pair of high resolution sensors 625 are positionedwithin housing 622. A positioner, dependent on the WFOV imagers thendirects the stereo imager 620 to a region of interest for additionalanalysis, such as region of interest distance, length, height, and/orcolor. Calibration of the stereo imager provides information as toerrors in measuring distance for various distances. TABLE 1 summarizesthe calibration results, with a reference distance a turbine and actualdistance determined with a laser range finder. Avian wingspan isproportional to distance from the imager, so that any error in thedistance leads to a corresponding error in determined wingspan. Maximumobserved error, at a distance of 1.1 km, was 3.7%, which for a raptorwingspan of 77″ corresponds to an error of about 2.85″. This is withinacceptable tolerance without unduly impacting wingspan length as auseful parameter for avian classification.

Further stereo camera measurement accuracy is achieved by finding atarget point on the moving object for each imager of the stereo imager.The target point may be the centroid, with errors in the target pointtranslating to distance errors. A one pixel error may cause an up to 20%distance error. Common centroids, therefore, are computed to sub-pixelaccuracy by tuning both cameras similarly.

Other important aspects of the stereo vision system is a center ofgravity closer to the axis of motion for reduced wind loading effect,structural rigidity for the imaging elements, a mass of less than 6 kg,with one camera fixed and the other camera adjustable for improvedstereo alignment, and hydrophobic viewports for better imagingperformance in mixed weather conditions. For low temperature operation,heating elements may be provided.

A more detailed illustration of WFOV or first imaging system 610 isprovided in FIGS. 12-13 , panels A-C. Each first imager or WFOV imagingsystem 610 may contain first imagers that are WFOV sensors 615, such asa pair of WFOV cameras extending along axes indicated as dashed lines inFIG. 13 , panel B, having a separation angle 616 to provide desiredairspace coverage, such as an angle between about 50° and 70°, or about60°. The sensors may be cameras contained within WFOV housing 612. Inthis example, each WFOV imager 615 images about 65° at 800 m, so thatthe pair of WFOV imagers together images about 130° at 800 m. The pantilt unit is integrated for rapid tracking of multiple different objectsto maximize high resolution images from the second imaging system 620 ofdifferent birds and to avoid tracking a single object for long timeperiods. As desired, various setting are controllable to establishinitial tracking priorities, continued tracking priorities, and amaximum number of degrees to move to another target. Generally, theimagers comprise a lens portion and a sensor portion, selected toprovide the desired resolution and field of view for the application ofinterest.

Other aspects of the first imager system are that cameras may be in afixed position and set for accurate location/relocation, improvedimaging performance in variable weather conditions by tilting viewportsrelative to vertical to reduce obstruction and use of hydrophobiccoating to decrease water beading. There is a common enclosure for bothimagers and the imagers of the first imaging system may be mounted tothe same substrate of the stereo imager of the second imager system.This increases the stability of the pan tilt calibration and reduces thepotential for change/drift over time and environmental conditions.

A ground enclosure 650 (see, e.g., FIGS. 7 and 14-15 ) can be used tocontain ancillary equipment, particularly for systems that arestand-alone and independent from any other structure, such as a windturbine or radar and building structure. Exemplary ancillary equipmentincludes, computers, servers, controllers, power equipment, climatecontrol, and electronic controllers for interfacing with a wind farm forwind farm applications, including a mitigation or wind turbine bladespeed control. A/C to D/C power conversion may occur within the groundlevel enclosure, thereby further reducing the mass and thermal load onthe tower. D/C to D/C power conversion may occur within each imaging podto simplify power transmission cabling and reduce cabling cost. Thecables are sealed and positioned within the bulkhead or tower passage,facilitating quick and efficient disconnection of imaging pods from theimaging tower. For an eight video stream system (e.g., six WFOV and twohigh resolution cameras), processing power requirements is about 500Mbytes/sec, which may be handled by a single multi-core computer.

Example 5: Field Test Results

FIGS. 16A-16C are images of a raptor from the second imager, e.g., thehigh resolution stereo imager, illustrating different bird postures andtheir impact on observed wingspan. Accordingly, an important input inthe avian analysis detection and classification is a ratio of wingspan(w) to height (h), or vice versa, because those dimensions vary withavian posture. FIG. 16D illustrates a corresponding wing-span multiplieras a function of the ratio h/w. This provides advantages over a simpleheuristic method that estimates wingspan of birds oriented at variousangles relative to the optical system. The technique provided hereinwhere the ratio of w/h or h/w is used to obtain a wingspan multiplier isgenerally referred to as “pose-estimation”. Even without suchpose-estimation, the test system correctly identifies 92% of goldeneagles as large raptors from measure wingspan alone.

Referring to FIG. 17 , an avian detection test system is installed in awind farm as a stand-alone system, labelled as “observation tower” ortower 605 in FIG. 17 . In this test, the WFOV corresponds to about 120°,with 600 m, 800 m and 1 km distance from the imager illustrated, alongwith ground topography and wind turbine 201 locations. Additionalimagers may be employed to provide hemispherical coverage, includingwith additional 120° imager systems and/or additional “observationtowers” at distinct geographical locations to provide an appropriatedetection envelope with respect to each turbine. The test system isparticularly useful in obtaining comprehensive data sets for use inoptimization and validation of the detection system, thereby ensuringenhanced image quality and detection reliability. System durability inthe field is also assessed.

FIG. 18 is an image of a moving object identified as a large raptor. Thedata associated with this image is: frame number, width and height (asreflected by the box around the raptor), distance, time of detection,and statistical confidence level. FIG. 19 is an image of the raptor at alater time, reflecting the accurate tracking and different glide posturerelative to the imager as the raptor changes direction and position. Apaired high resolution and WFOV image of a raptor is provided in FIG. 20. The top panel is the high resolution image from a second imager andthe bottom panel a WFOV image from the first imager. Successful trackingof a plurality of moving raptors 230 and 233 is illustrated in theimages of FIG. 21 , panels (i)-(v). Both raptors are located “above” theturbines in panel (i), with raptor 233 decreasing in altitude asreflected in panel (ii). The importance of turbine masking isillustrated in panel (iii), with raptor 233 in at least visual proximityto a wind turbine. The raptor 233 is successfully tracked during flight,including as it increases vertical distance from the ground, asillustrated in panels (iii)-(v).

The test system also facilities collection of images suitable for futureclassification (post-collection processing and analysis). With thesystem, 3,890 tracks are recorded, including 148 high-resolution videosof eagles. Of those videos, 26 are within a target stereo range of 300 mto 1 km from the imager, with 92.3% correct classification by wingspanalone. Further improvement is expected with additional avianidentification parameters, including color analysis. The system alsocaptured 8 high resolution videos of non-eagle avians.

STATEMENTS REGARDING INCORPORATION BY REFERENCE AND VARIATIONS

All references throughout this application, for example patent documentsincluding issued or granted patents or equivalents; patent applicationpublications; and non-patent literature documents or other sourcematerial; are hereby incorporated by reference herein in theirentireties, as though individually incorporated by reference, to theextent each reference is at least partially not inconsistent with thedisclosure in this application (for example, a reference that ispartially inconsistent is incorporated by reference except for thepartially inconsistent portion of the reference).

The terms and expressions which have been employed herein are used asterms of description and not of limitation, and there is no intention inthe use of such terms and expressions of excluding any equivalents ofthe features shown and described or portions thereof, but it isrecognized that various modifications are possible within the scope ofthe invention claimed. Thus, it should be understood that although thepresent invention has been specifically disclosed by preferredembodiments, exemplary embodiments and optional features, modificationand variation of the concepts herein disclosed may be resorted to bythose skilled in the art, and that such modifications and variations areconsidered to be within the scope of this invention as defined by theappended claims. The specific embodiments provided herein are examplesof useful embodiments of the present invention and it will be apparentto one skilled in the art that the present invention may be carried outusing a large number of variations of the devices, device components,methods steps set forth in the present description. As will be obviousto one of skill in the art, methods and devices useful for the presentmethods can include a large number of optional composition andprocessing elements and steps.

When a group of substituents is disclosed herein, it is understood thatall individual members of that group and all subgroups, are disclosedseparately. When a Markush group or other grouping is used herein, allindividual members of the group and all combinations and subcombinationspossible of the group are intended to be individually included in thedisclosure. Every combination of components or steps described orexemplified herein can be used to practice the invention, unlessotherwise stated.

Whenever a range is given in the specification, for example, a volumerange, a zoom range, a number range, a distance range, a percentagerange, all intermediate ranges and subranges, as well as all individualvalues included in the ranges given are intended to be included in thedisclosure. It will be understood that any subranges or individualvalues in a range or subrange that are included in the descriptionherein can be excluded from the claims herein.

All patents and publications mentioned in the specification areindicative of the levels of skill of those skilled in the art to whichthe invention pertains. References cited herein are incorporated byreference herein in their entirety to indicate the state of the art asof their publication or filing date and it is intended that thisinformation can be employed herein, if needed, to exclude specificembodiments that are in the prior art. For example, when composition ofmatter are claimed, it should be understood that compounds known andavailable in the art prior to Applicant's invention, including compoundsfor which an enabling disclosure is provided in the references citedherein, are not intended to be included in the composition of matterclaims herein.

As used herein, “comprising” is synonymous with “including,”“containing,” or “characterized by,” and is inclusive or open-ended anddoes not exclude additional, unrecited elements or method steps. As usedherein, “consisting of” excludes any element, step, or ingredient notspecified in the claim element. As used herein, “consisting essentiallyof” does not exclude materials or steps that do not materially affectthe basic and novel characteristics of the claim. In each instanceherein any of the terms “comprising”, “consisting essentially of” and“consisting of” may be replaced with either of the other two terms. Theinvention illustratively described herein suitably may be practiced inthe absence of any element or elements, limitation or limitations whichis not specifically disclosed herein.

One of ordinary skill in the art will appreciate that components,devices, algorithms, and processes other than those specificallyexemplified can be employed in the practice of the invention withoutresort to undue experimentation. All art-known functional equivalents,of any such components, devices, algorithms, and processes are intendedto be included in this invention. The terms and expressions which havebeen employed are used as terms of description and not of limitation,and there is no intention that in the use of such terms and expressionsof excluding any equivalents of the features shown and described orportions thereof, but it is recognized that various modifications arepossible within the scope of the invention claimed. Thus, it should beunderstood that although the present invention has been specificallydisclosed by preferred embodiments and optional features, modificationand variation of the concepts herein disclosed may be resorted to bythose skilled in the art, and that such modifications and variations areconsidered to be within the scope of this invention as defined by theappended claims.

TABLE 1 Calibration results for high resolution sensors ReferenceDistance Average Error Worst Error Worst Error [m] [m] [m] [%] 677 11.54−18.55 −2.7% 894 13.88 25.80 2.9% 1,104 20.53 41.06 3.7%

We claim:
 1. A detection system for detecting a flying object in anairspace comprising: a plurality of first imagers that are fixablypositioned in each of a plurality of distinct alignment directions, eachof the plurality of first imagers having a wide field of view fordetecting a moving object, wherein the plurality of first imagers arearranged in a spatial configuration to provide from the plurality ofalignment directions a substantially complete hemispherical coverage; aplurality of second imagers each having a high zoom, wherein theplurality of second imagers is a stereo imager; a positioner operablyconnected to the stereo imager for positioning the stereo imager toimage the moving object detected by the plurality of first imagers,wherein the positioner moves an alignment direction of the stereo imagerbased on an output from at least one of the plurality of first imagerswithout moving any of the plurality of first imagers; a processoroperably connected to receive image data from the plurality of firstimagers, the plurality of second imagers, or both to identify the movingobject that is an artificially-constructed flying object or a flyingavian based on said image data; wherein the plurality of first imagersarranged in the plurality of distinct alignment directions provides full360° and the substantially complete hemispherical coverage for detectionof the moving object in any direction relative to the detection system.2. The detection system of claim 1, wherein the substantially completehemispherical coverage provides coverage for a volume of airspace havinga detection distance from said first imager that is greater than orequal to 0.6 km and less than or equal to 2 km.
 3. The detection systemof claim 1, wherein said artificially-constructed flying object is anairplane, helicopter, hot-air balloon, or other man-made object.
 4. Thedetection system of claim 1, wherein said processor identifies an outputof a subset of pixels of said first imager or said second imagercorresponding to said moving object having a boundary parameter or pixelpattern indicative of an artificially-constructed object or a flyingavian.
 5. The detection system of claim 4, wherein said output of saidsubset of pixels is an array of intensity values and/or is a timevarying output.
 6. The detection system of claim 4 wherein saidprocessor analyzes said output of said subset of pixels, using one ormultiple algorithms in combination, such as pattern recognition, edgedetection and/or boundary parameter analysis, to determine if saidmoving object is a flying artificially-constructed object.
 7. Thedetection system of claim 4, wherein said boundary parameter correspondsto an edge boundary signature that is: straight or smooth for theartificial object; or that is not highly straight or smooth for theavian.
 8. The detection system of claim 1, wherein said processoridentifies an output of a subset of pixels of said first imager or saidsecond imager corresponding to said moving object, and wherein saidprocessor analyzes said output to identify the presence of one or morethreshold identification attributes that is a color parameter.
 9. Thedetection system of claim 4, wherein said processor analyzes said outputof said subset of pixels via a pattern recognition algorithm.
 10. Thedetection system of claim 4, wherein the processor analyzes said outputof said subset of pixels from a plurality of frames of said image data,wherein said subset of pixels spatially moves with time and saidmovement with time is used to determine a trajectory of said output ofsaid subset of pixels.
 11. The detection system of claim 10, whereinsaid trajectory comprises positions, distances, velocities, directionsor any combination thereof at a plurality of times.
 12. The detectionsystem of claim 11, further comprising determining a predictivetrajectory corresponding to a future time interval.
 13. The detectionsystem of claim 1, wherein said second imager has a resolution that isselected from a range that is greater than or equal to 1 cm per pixeland less than or equal to 4 cm per pixel and/or said high zoom isselected from a range that is greater than or equal to 10x and less thanor equal to 1000x.
 14. The detection system of claim 1, wherein saidfirst imager, said second imager, or both said first and the secondimagers detect a wavelength range corresponding to light in the visibleor infra-red spectrum.
 15. The detection system of claim 1, configuredto simultaneously identify a plurality of moving objects.
 16. Thedetection system of claim 1, used to decrease incidence of avian strikeson an airplane around airport runways.
 17. The detection system of claim1, wherein said airspace corresponds to an airspace volume along adirection of airplane traffic movement.
 18. The detection system ofclaim 1 that is stationary.
 19. The detection system of claim 1 that ismounted to a moving vehicle.
 20. The detection system of claim 1,wherein said positioner comprises a motorized pan and tilt headconnected to said second imager for moving an alignment direction ofsaid second imager based on an output from said first imager.
 21. Thedetection system of claim 1, wherein said first imager, said secondimager, or both said first and second imagers are cameras.
 22. Adetection system for detecting a flying object in an airspacecomprising: a plurality of first imagers that are fixably positioned ineach of a plurality of distinct alignment directions, each having a widefield of view for detecting a moving object, wherein the plurality ofdistinct alignment directions covers all approaches to the detectionsystem; a stereo imager comprising a pair of second imagers eachindependently having a high zoom; a positioner operably connected to thestereo imager for positioning said stereo imager to image said movingobject detected by the plurality of first imagers, wherein thepositioner moves an alignment direction of the stereo imager based on anoutput from the first imagers without moving any of the plurality offirst imagers; and a processor operably connected to receive image datafrom said plurality of first imagers, said stereo imager, or both and todetermine a position and trajectory of said moving object, therebyidentifying a moving object that is a flying avian or anartificially-constructed object based on image data from the pluralityof first imagers, the stereo imager, or both the plurality of firstimagers and the stereo imager.
 23. The detection system of claim 22,providing substantially complete hemispherical coverage of said airspacesurrounding the detection system.
 24. The detection system of claim 22,wherein said airspace is around an airport runway.
 25. The detectionsystem of claim 24, wherein said moving object is anartificially-constructed object.
 26. The detection system of claim 22,further comprising: a plurality of wide field of view systems, each widefield of view system comprising a pair of said first imagers forming analignment angle with respect to each other to provide a field of viewangle for each wide field of view system that is greater than or equalto 90° and less than or equal to 180°, wherein said plurality of widefield of view systems in combination provides 360° imaging coveragearound said detection system.
 27. The detection system of claim 26,further comprising: a tower interface for connecting each of the widefield of view systems and the stereo imager system to a tower.
 28. Amethod of detecting a flying object in an airspace, the methodcomprising the steps of: imaging the airspace surrounding the detectionsystem of claim 1; obtaining one or more threshold identificationattributes for an output of a subset of pixels from the imaging step;analyzing the one or more threshold identification attributes toidentify a moving object of interest; obtaining one or moreidentification parameters for the moving object of interest; comparingthe one or more identification parameters to a corresponding one or morereference identification parameters to identify the moving object ofinterest as an avian or an artificially-constructed object; and whereinthe method detects the moving object of interest within the airspacehaving a volume equivalent to an average-equivalent hemisphere with anaverage radius selected from a range that is greater than or equal to0.5 km and less than or equal to 1.2 km.
 29. The method of claim 28,wherein the imaging step comprises identifying an output of a subset ofpixels that is an array of light intensity values.
 30. The method ofclaim 28, wherein the imaging comprises obtaining a wide field of viewwith a first imager and optically zooming in on the moving object ofinterest with a second imager, wherein the second imager is used todetermine a distance of the moving object of interest from the imagingsystem.
 31. The method of claim 28, for detecting a moving object thatis an artificially created object.
 32. The method of claim 28, whereinthe imaging step further comprises obtaining a plurality of images atdifferent times and determining a trajectory of the output of the subsetof pixels.
 33. The method of claim 30, wherein the distance isdetermined using the stereo imager that is positioned to image themoving object.
 34. The method of claim 31, wherein the analyzing step isvia a pattern recognition algorithm.
 35. The method of claim 28, whereinthe one or more threshold identification attributes is selected from thegroup consisting of distance, trajectory, boundary parameter, boundaryshape, edge boundary characteristic, pixel spacing, pixel intensity,pixel color, intensity gradient, time evolution parameter, and anycombination thereof.
 36. The method of claim 35, wherein the one or morethreshold identification attributes is a boundary parameter.
 37. Themethod of claim 36, further comprising the step of identifying a movingobject as corresponding to an artificially-constructed object byidentifying at least a portion of the boundary parameter as having ashape indicative of an artificially-constructed object.
 38. The methodof claim 37, wherein the boundary parameter comprises an edgestraightness parameter indicative of the artificially constructedobject.
 39. The method of claim 28, wherein the comparing step comprisesa pattern recognition algorithm.
 40. The method of claim 28, furthercomprising the step of obtaining a predictive trajectory of the flyingobject.
 41. The method of claim 40, used at an airport.
 42. The methodof claim 28, further comprising implementing an action step, wherein theimplementing step comprises one or more of: providing an alert to aperson; emitting an alarm; triggering a count event; triggering adeterrent to encourage movement of the flying object out of the airspacesurrounding the imaging system; recording an image or video of theobject flying through the airspace surrounding the imaging system. 43.The method of claim 42, further comprising the step of defining anaction implementation airspace having an average action distance that isless than the average-equivalent radius of the substantiallyhemispherical airspace surrounding the imaging system, wherein theaction implementation is implemented for a flying object that is: withinthe substantially hemispherical airspace and having a trajectory towardthe action implementation airspace; or within the action implementationairspace.
 44. A detection system for detecting a flying object in anairspace surrounding a wind turbine comprising: a plurality of imagingsystems, each imaging system comprising: a plurality of first imagersthat are fixably positioned in each of a plurality of distinct alignmentdirections, each of the first imagers having a wide field of view fordetecting a moving object, wherein the plurality of first imagers arearranged in a spatial configuration to provide from the plurality ofalignment directions a substantially complete hemispherical coverage; aplurality of second imagers, each having a high zoom, wherein the secondimager is a stereo imager; wherein the plurality of first imagers andthe plurality of second imagers determines a position and a trajectoryof a flying object in the airspace; a positioner operably connected tothe stereo imager for positioning the stereo imager to image the movingobject detected by the plurality of first imagers, wherein thepositioner moves an alignment direction of the stereo imager based on anoutput from the first imagers without moving any of the plurality offirst imagers; a processor operably connected to receive image data fromany of the plurality of first imagers, the plurality of second imagers,or both, and to identify the moving object that is a flying avian or anartificially-constructed flying object based on said image data; whereinthe plurality of first imagers are positioned relative to each other toprovide substantially complete hemispherical coverage of said airspacesurrounding the wind turbine; and a controller that receives output fromthe processor, the controller operably connected to the wind turbine fordecreasing or stopping wind turbine blades for a flying objectidentified as at risk of otherwise striking a moving blade of the windturbine.
 45. The detection system of claim 1, wherein the stereo imagercomprises a pair of the second imagers each independently having a highzoom.
 46. The detection system of claim 27, wherein the stereo imagercomprises a pair of second imagers each independently having a highzoom, said avian detection system further comprising: at least threewide field of view systems, each providing a field of view between 120°and 140°; a ground enclosure containing ancillary equipment electricallyconnected to said imagers by cables that run through an inner passagewithin the tower; and a lightning mitigation system extending from thetower top, wherein the imagers are positioned so as to image airspacearound the tower without optical obstruction by the lightning mitigationsystem.
 47. The detection system of claim 1, wherein the processoridentifies the flying avian as a flying avian species of interest. 48.The detection system of claim 1, wherein the detection system isconnected to a wind turbine.
 49. The detection system of claim 1,wherein the stereo imager comprises a pair of imagers to determine aposition and a distance of the moving object from the detection system.50. The detection system of claim 1, wherein the stereo imager comprisesa pair of imagers to determine a distance and position of the movingobject from a wind turbine.
 51. The detection system of claim 50,wherein the moving object is a flying avian, and the processor comparesa physical parameter determined by the stereo imager to classify theflying avian as an avian species of interest.